I have the same issue.
Here is a short video showing it happening.

I have a fresh install as per instructions on https://github.com/danielfreer/raspberrypi5-retropie-setup?tab=readme-ov-file

Pi 5 connected via hdmi for audio.

I've tried the fixes that worked for some above, but nothing changes.
as per @mitu request I have run the amixer/aplay commands

pi@retropie:~ $ amixer
pi@retropie:~ $ aplay -1
aplay: invalid option -- '1'
Try aplay --help' for more information. pi@retropie:~ $ aplay -1 aplay: invalid option -- '1' Try aplay --help' for more information.
pi@retropie:~ $ aplay -L
null
Discard all samples (playback) or generate zero samples (capture)
default
Default Audio Device
sysdefault
Default Audio Device
hw:CARD=vc4hdmi0,DEV=0
vc4-hdmi-0, MAI PCM i2s-hifi-0
Direct hardware device without any conversions
plughw:CARD=vc4hdmi0,DEV=0
vc4-hdmi-0, MAI PCM i2s-hifi-0
Hardware device with all software conversions
default:CARD=vc4hdmi0
vc4-hdmi-0, MAI PCM i2s-hifi-0
Default Audio Device
sysdefault:CARD=vc4hdmi0
vc4-hdmi-0, MAI PCM i2s-hifi-0
Default Audio Device
hdmi:CARD=vc4hdmi0,DEV=0
vc4-hdmi-0, MAI PCM i2s-hifi-0
HDMI Audio Output
dmix:CARD=vc4hdmi0,DEV=0
vc4-hdmi-0, MAI PCM i2s-hifi-0
Direct sample mixing device
hw:CARD=vc4hdmi1,DEV=0
vc4-hdmi-1, MAI PCM i2s-hifi-0
Direct hardware device without any conversions
plughw:CARD=vc4hdmi1,DEV=0
vc4-hdmi-1, MAI PCM i2s-hifi-0
Hardware device with all software conversions
default:CARD=vc4hdmi1
vc4-hdmi-1, MAI PCM i2s-hifi-0
Default Audio Device
sysdefault:CARD=vc4hdmi1
vc4-hdmi-1, MAI PCM i2s-hifi-0
Default Audio Device
hdmi:CARD=vc4hdmi1,DEV=0
vc4-hdmi-1, MAI PCM i2s-hifi-0
HDMI Audio Output
dmix:CARD=vc4hdmi1,DEV=0
vc4-hdmi-1, MAI PCM i2s-hifi-0
Direct sample mixing device